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In  this  paper  we  study  scaling  rules  and  round-off  noise  variances. in  a 
fixed  point  implementation  of  the  Kalman  predictor  for  an  ARMA  time  series 
observed  noise- free.  The  Kalman  predictor  is  realized  in  a  fast  form  that 
uses  the  so-called  fast  Kalman  gain  algorithm.  The  algorithm  for  the  gain 
is  fixed  point. 

Scaling  rules  and  expressions  for  rounding  error  variances  are  derived. 
The  numerical  results  show  that  the  fixed  point  realization  performs  very 
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closely  to  the  floating  point  realization  for  relatively  low-order  ARMA 
time  series  that  are  not  too  narrowband. 

The  predictor  has  been  implemented  in  16-bit  fixed  point  arithmetic 
on  an  INTEL  8086  microprocessor,  and  in  16-bit  floating  point  arithmetic 
on  an  INTEL  8080.  Fixed  point  code  was  written  in  ASSEMBLY  language  and 
floating  point  code  was  written  in  FORTRAN.  Experimental  results  were 
obtained  by  running  the  fixed  and  floating  point  filters  on  identical  data 
sets.  All  experiments  were  carried  out  on  an  INTEL  MDS  230  Development 
System. 
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la  this  paper  ft  stvdy  scaling  rales  and 
round-off  noise  variances  in  a  fixed  point 
implementation  of  the  falaan  predictor  for  an 
ARMA  tine  series  observed  noise-free.  The  Kalian 
predictor  is  realised  in  a  fast  fori  that  ases 
the  so-called  fast  Kalian  gain  algorithm.  The 
algorithm  for  the  gain  is  fixed  point. 

Soiling  rales  and  expressions  for  roanding 
error  variances  ere  derived.  The  aumtriesl 
resalts  show  that  the  fixed  point  realisation 
perforin  very  closely  to  the  floating  point 
realisation  for  relatively  low-order  ARMA  tine 
series  that  are  not  too  narrowband. 

The  predictor  has  been  implemented  in  ld-hit 
fixed  point  arithmetic  on  an  INTEL  8088 
microprocessor,  and  in  Id-bit  floating  point 
arithmetic  on  an  INTEL  8080.  Fixed  point  code 
was  written  in  ASSEMBLY  language 41  and  floating 
point  code  was  written  in  FORTRAN.  Experimental 
resalts  were  obtained  by  running  the  fixed  and 
floating  point  filters  on  identical  data’ sets. 
All  experiments  were4  carried  oat  on  an  INTEL  IDS 
23b  Development  System. 


Introdnotlon  ^ 

Finite-dimensional  Gaussian  time  series  have 
stationary  Markovian  state-space  descriptions. 
Is  sach  descriptions  the  initial  conditions  are 
— ltivariate  normal  and  state  variables  are 
predicted  valaes  of  the  series  based  on  an 
infinite  past  of  observations.  The  linear 
filtering  problem  is  one  of  estimetlng  the  state 
at  time  t  based  on  observations  ap  to  time  t  and 
the  prediction  problem  is  one  of  predicting  the 
state  at  time  tel  based  on  observations  ap  to 
time  t. 

Corresponding  to  the  Markovian 
representation  is  the  innovations  representation. 
The  essential  characteristic  of  this 
nonstationary  representation  is  that  it  may  be 
ased  to  synthesise  a  time-series,  starting  from 
sero  initial  conditions,  whose  second-order 
statistics  match  the  statistics  of  the  original 
time  series.  The  states  are  predicted  valnes  of 
the  time  series  based  on  a  finite  past  of 
observations.  Osing  this  representation,  the 
Kalman  predictor  may  be  written  down  from 

This  paper  was  presented  at  the  22nd  I  ERE 
Conference  on  Decision  and  Control,  San  Antonio, 
11  (December  14-ld,  1983).  The  work  was 
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inspection  as  the  causal  and  stable  inverse  of 
the  representation.  The  so-called  Kalman  gain  in 
the  innovations  representation  (or  equivalently 
in  the  Kalman  predictor)  may  be  associated  either 
with  the  Levinson  recursions  for  factoring  the 
inverse  of  the  correlation  matrix  for  the  time 
series  or  with  the  LeRoux-Gueguen  recursions  for 
factoring  the  correlation  matrix  itself.  The 
latter  association  leads  to  a  fixed  point 
algorithm  for  computing  Kalman  gains.  This  so- 
called  fast  algorithm  products  s  fast  Kalman 
filter. 

In  this  paper  we  present  results  from  s 
study  of  fast  Kalman  predictors,  implemented  in 
floating  point  and  in  fixed  point  arithmetic,  for 
autoregressive  moving  average  time  series.  Mors 
extensive  results  of  this  study,  for  aolsy  and 
noise-free  filtering  and  prediction,  may  be  found 
in  the  thesis  of  Signrdsson  (1). 

In  onr  summary  of  results  for  Kalmaa 
filtering  wa  draw  heavily  npon  the  work  of  Morf, 
Kailath,  And# r ton,  and  Moore.  Sea  (2)  and  (3) 
for  oar  previous  references  to  the  appropriate 
literature.  In  onr  derivation  of  scaling  rules 
sad  expressions  for  rounding  error  variances  we 
adapt  tke  stationary  raaulta  of  Jackson  [4)  and 
Mull  is  and  Roberts  [51  to  oor  nonstst ionary 
p rob  lam. 

Signal  Mods  Is  for  Stationary  ARMA  Time  Series 

A  taro-naan,  second-order  stationary,  time 
ssrias  (y(t>)  is  said  to  be  autoregressive  moving 
average  (ARMA)  if  the  entries  y(t)  in  the  time 
aariaa  obey  this  rtcarsion  for  all  t: 


W  'R 

^  a(n)y(t-n)  -  ^  b(a)iU-a) 

n-0  n«0 


E  a(t)a(t+a)  -  o  6(a) 


a  ■  h  -  1  8(a):  Kroaecker  delta 

o  o 


The  Markovian  raprasantation  for  ty(t>)  is 
an— arixed  in  the  following  aquations,  where  a  - 
max(p.q),  and  where  for  parpoaas  of  illustration 
wa  have  assumed  q  >  p. 


tw 


lutt  lt>«tl»il 


State  equations 


■(0/-1J  :  N<0»a)  j  m( t)  -  N(0.o*> 


fit)  »  e*  ♦  u<t) 

On  it  Nl»i  jgitgaM 


«'AMfc(l),  t>0 


State  ConriiMt 


»<»)  -  A*Q 

Q  -  AOA*  ♦  e2hU)h'<l) 


Output  Comiwt 


nCt+l/t)  *  AxU/t-1)  *  kU)  *(t) 

* (0/-1 )  -  0  J  n(t)  :  N<0,v<t)> 
j(t )  «  c*  x ( t/t-1 )  ♦  n(t) 

Quit  Paisa  losponso 

0.  n<0 

hl(n)  -  1.  n *0 

c'A®’1  k(  t) .  n>0 

Stats  Covarisnci 

Rt(n)  -  AnQ ( t ) 

Q(t*l)  -  AQ(t )A*  ♦  v(t)k(t)k'(t) 


r(t)  -  c**(t)  c  ♦  •  b(t) 


Design  Equations 


c  -  *  k(l) 


b(l)1  f  a(l)l  fb(l)l 


a(l)  1  0  I  I  fed)  j  a(2)  I  b<2) 


iCp-1)  ...  1  a(p) 

»  a(p-l)  lj  l  h(n)  J  l  0  j  lb(q). 


In— vat  ion  a  la 


Tkt  ilia  kikial  the  innovations 
tifriHstitiof  fof  (y(t))  la  to  replace  tha 
atat ionary  initial  conditions,  distributed  as 
s(0/-l)  :  0(0,  Q)  with  tbo  —  natationary  initial 

•salitissi  *(0/-l)  •  0,  to  roplano  tbo  stationary 
inpot  vootor  b(l)  by  "a  nonstationary  Kalnsn  pain 
vootor  |(t).  and  to  replace  tbo  stationary  1,1.0. 
sop— so o  (n(t)l,  distributed  aa  o(t)  :  NCO,  e2), 
with  tbo  nonstat ionary  1,1,4.  innovations 
sop— —a  (n(t>),  distributed  as  n(t)  N(0, 
v(t))«  Tbo  trial  la  to  obooso  tbo  Kalnan  gain 
kit)  and  tbo  innovations  variance  v(t)  correctly, 
hi  in— vat  ions  roproaontat  ion  Is  sn—wrlxod 
bo  lev.  It  is  — rtb  —ting  that  k(t)  ->hd>,  v(t) 
-le2,  and  0(t)  -10  aa  t  ->•. 


Output  Covariance 


rt(n)  -  c'iMnlc  ♦  v(t)bl(n) 


Poalan  Equations 


k<U  vU)  -  M<k  -  Q(t>>c  ♦  o  hil )i  010)  -  0 
Q  -  AQA*  ♦  o2b(l)b* (1) 

v(t)  •  r (o)  -  c'  Q(t)c  ;  r(o)  *  c 'Qc  ♦  a1 

In  tbs  so  aquations,  bt(n)  it  tho  rosponso  of 
tbo  systoa  of  aquations  at  t too  t+u  to  an  inpulas 
applied  at  tins  t,  t*(n)  is  tbo  stato  convarianco 
bstvoon  x(t)  and  x(t+n),  and  r*(n)  is  tbo  ontpot 
covariance  between  y(t)  and  y(t*n).  If  rt(n)  is 
to  equal  r(n),  to  natch  the  second  order 
properties,  then  k(t)  and  v(t)  oust  bo  chosen  as 
above  in  tbs  design  aquations. 

la loan  Predictor 

In  tbo  innovations  representation,  identify 
e'*(t+l/t)  os  the  prediction  y(t+l/t).  Then  the 
ta 1 nan  fitter  equations  be cone  tinply  a  rewriting 
of  the  innovations  representation: 


y  ( td/t )  «•  c'  «(t*l/t) 

x(td/t>  -  Ax(t/t-l)vk(t)(y(t)-c'x<t/t-l)) 
y(t)  *  y(t) 


a 


2 


gut  Algoritba  for  k(t) 

The  $«ii  Mt)  My  b#  calculated  as  outlined 
u4«r  tba  laioution  equations  by  loUiat  a 
systea  of  fticatti  i<sitios».  As  alternative  Is 
to  sots  that  tba  gala  is  related  to  tba  tiae 
varying  iapalse  response  of  the  iaaovatloas 
rapresaatatioa: 


bSa)  •  c'A1  *k(t)  ,  a>0 


This  aeaas  tba  eleaeats  of  k(t)  say  ba  read  oat 
as  follows: 


k‘U>  -  k(t> 


J  fk<«>  -  |«  j"'1/*!2 


where  6  is  a  parameter  that  say  be  iacressed  to 
force  the  norn  of  fk  to  be  snail*  aad  thereby 
decrease  the  probability  of  overflow.  The 
parameter  «r  is  the  variance  of  the  aoisy 
excitation  of  the  filter. 

la  the  Kalaan  predictor  equation  we  nay 
scale  tba  state  vector  x(t/t-l)  by  a  diagonal 
seal  lag  aatria  S : 


S  -  diag  <SU> . S(e>) 


x<t*l/t)  -  S_lAS  x(t/t-l>  ♦  S_1k(t)  a ( t ) 


y( t/t-1 )  *  c*  S  * < t/t-1 ) 


As  tba  correlation  aatrix  for  y(t)  is  related  to 
the  tiae  varyiag  iapalse  responses*  we  aay  write 


The  I2  scaling  rnles  for  the  states  of  the  Kalaan 
predictor  are 


where  the  t  x  t  aatrices  t,V,  and  H  are  defined 
as  follows: 


s''nit)l  $'T  -  |>  2"'1/*!2 


The  state  variance  Q(t)  aay  be  coaputed 
recursively  as  outlined  previously,  or 
nonrecursive ly  as 


?  -  diag  ( v (o ) , . . . v ( t ) ) 


H  -  bw(l)  b*(0) 


(t)  b  (t-1) 


This  aaaaa  the  correlation  aatrix  aay  ba  factored 
with  the  fast  iapalse  response  algoritba  of 
Lelomx-Gaegaea  (6)  to  obtain  f*  aad  a-diaeasioaal 
colaaas  of  I  aay  be  picked  off  to  obtain  k(t). 

teal lag  and  lonndlai  in  the  Kalaan  Tredlotor 

Let  C  denote  quantisation  step  sixe  and 
the  boa  ad  on  the  aaxiaaa  aagaitnde  that 
any  be  represeated  in  an  n-bit.  signed  binary 
representation.  The  problea  in  a  finite-word 
leagtb  realisation  of  the  Kalaan  predictor  is  to 
••ale  variables  so  that  the  probability  of 
overflow  is  snail. 


A  filter  is  said  to  be  K  scaled  if 


t 

0<t)  -  v ( t )  Y  A-’1  |(t-a)  (A*-1  k( t-n) ) ' 
n-1 

The  diagonal  terns  of  Q(t)  converge 
stoaotonieai  ly  to  the  npper  Unit  Q  *  AQA '  ♦ 
w  btilb*(l)«  A  practical  procedure  is  to 
replace  Q(t)  by  Q  to  obtain  the  scaling  rale 

S(k)  »  tQ]fc*/2  h/2*'1  (k-1,2, , .  .a) 


This  resnlt  for  stationary  state  space  filters  is 
doe  to  Mall  is  and  loberts  (5). 

The  algoritba  for  the  gain  need  not  be 
scaled  becanse  it  is  fixed  point:  all  internal 
variables  are  boanded  by  natty  in  aagnitnde. 


The  apdatiag  of  the  state  vector  in  the 
Kalaan  predictor  requires  one  aeltiply  for  the 
f|j*t  (n-1)  eleaents  sad  (n+l)  anltiples  for  the 
a  eleaeat.  By  associating  a  sequence  of 
i.i.d.,  variance  £2/12,  randoa  variables  with 
each  fixed  point  nultiply,  we  generate  s  aean 
sero,  variance  N  randoa  vector  n(t)  each  tiae  we 
nodate: 


\ 


2<t+l/t>  -  S_1AS  s<t/t-l)*S~i(t)u(t>+a<t> 
B  a(t)a'(t*k)  -  N6U) 

ri  1 


l  <n*l) 

Tka  state  viriaaci  ii  now 


Q(t*l)  -  (»*1AS)0(t)(S"1AS),^(t)k(t)k,(t)*N 
O-(S1AS)0(S_1AS),^«2k<1^'(l)  ♦  N 

Wiaaglaal  hgtfigaila 

Refer  to  tko  abstract  for  a  summary  of  hov 
all  experiments  vara  conducted.  that  folio**  is 
a  briaf  annotation  of  Figaro*  1  through  4. 

Experiments  vara  conducted  by  goaaratiag 
raaliaatioas  of  a  statioaary  t iaa  saria*  froa  the 
model 

i(l)  .  1-1.75,'1  ♦  Q.»,~* _ 

-1  -4 
1  ♦  1.21s  *  -  0.4550*  * 

Figaro  1  lllastrata*  k(t),  a  3il  vector, 
computed  vith  floatiag  poiat  arithmetic  (carva* 
1-3)  aad  vith  fiiad  poiat  arithmetic  (carva*  4- 
<)•  Figara  2  illaatrata*  tha  iaaovatioa  variaaca 
v(t)  aoapatad  ia  floatiag  poiat  (carva  1)  aad  ia 
fiiad  poiat.  Thasa  tvo  carva*  illustrate  that 
tha  data-iadapaadaat  fa»t  Kalman  gala  calculation 
awy  ba  practically  competed  ia  fixed  poiat  aaiag 
Id-bit*. 

Figaro  3  lllastrata*  pradictioas  y(t/t-l)  la 
tha  floatiag  poiat  realisation  (curve  1)  aad  ia 
tha  scaled,  fiiad  poiat  realisation  of  the  Kalaaa 
predictor. 


Figara  4  illustrates  tha  corresponding 
iaaovatioa*  scguences.  Tha  floatiag  point  and 
fiiad  poiat  predictors  vara  ran  over  tha  saaa 
real lsat ioa  of  tha  t iaa  series. 
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FIGURES 


Figure  1.  Tha  Kalman  predictor  gain  vector  k(t) 
calculated  using  floating  point  arithmetic 
(curves  1-3)  aad  fixed  point  arithmetic  (curves 
4-4). 
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